Mid-Technology  Readiness  Level  Technology  (TRL) 
Maturation  Planning,  TRL  Criteria  Setting  and 
Assessment  in  Support  of  In-Space  Technology 

Validation 

2007  Technology  Maturity  Conference 
Founders  Inn,  Virginia  Beach,  VA 

Christopher  M.  Stevens 
John  F.  Stocky 

September  12,  2007 
Jet  Propulsion  Laboratory 
California  Institute  of  Technology 


Form  Approved 
OMB  No.  0704-0188 


Report  Documentation  Page 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

12  SEP  2007 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2007  to  00-00-2007 

4.  TITLE  AND  SUBTITLE 

New  Millennium  Program.  Mid-Technology  Readiness  Level  Technology 
(TRL)  Maturation  Planning,  TRL  Criteria  Setting  and  Assessment  in 
Support  of  In-Space  Technology  Validation 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

California  Institute  of  Technology  Jet  Propulsion  Laboratory  ,4800  Oak 
Grove  Dr, Pasadena, CA, 91 109 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADM002182.  Presented  at  the  AFRL  Technology  Maturity  Conference  held  in  Virginia  Beach,  VA 
on  11-13  September  2007. 

14.  ABSTRACT 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 
OF  PAGES 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

22 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Discussion  Points 


*  Introduction 

-  Description  of  the  New  Millennium  Program 

-  Increasing  technological  maturity  in  a  project 
environment 

-  Approach  to  judging  technological  maturity 

*  NMP  Approach  to  Assessing  Technological 
Maturity 

-  Guidelines 

-  Mechanism  for  implementing  guidelines 

*  Approach  as  applied  to  ST  8  and  ST  9 

*  Summary 


AFRL  TMC  2007 


2 


Program  Objectives  and 

Purpose 

•  Accelerate  the  incorporation  of  advanced  technologies  into  future  NASA 
science  missions 

-  Derive  needed  capabilities  and  technical  requirements  from  NASA  science  mission 
planning  (“technology  pull”) 

-  Conduct  technology  maturation  and  validation  in  low  cost  NMP  project  rather  than  during 
science  mission  development 

-  Capitalize  on  investments  being  made  in  U.S  technological  capabilities 

-  Provide  infusion  path  with  validated  performance  scalable  to  intended  uses 

•  Balance  requirements/scope/cost  to  remain  “affordable” 

•  Focus  on  technologies  (“breakthrough  technologies”)  that  will  provide  a 
significant  (~order  of  magnitude)  step  forward  in  performance  and/or 
reduction  in  cost 

-  Enable  new  capabilities  to  meet  Earth  and  Space  Science  needs 

-  Target  technology  capabilities  that  are  broadly  applicable  to  future  needs 

-  Reduce  risk  and  costs  of  future  missions 

•  Select  and  mature  advanced  technologies  for  validation  in  the  space 
environment,  when  flight  validation  is  required  to  validate  performance  and 
reduce  performance  and  development  risks  to  the  first  users 

-  Objective  is  to  validate  the  technology  advance,  not  the  widget 

-  Intent  is  for  technologies  to  have  a  much  wider  domain  of  applicability  than  the 
specific  configuration  flown 
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Flight  Validation  Criteria 


Technologies  to  be  validated  in  space  must  meet  one  or  more  of 
the  NMP  Flight  Validation  Criteria 
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New  Millennium  Program 

Technology  Readiness  Levels  (TRLs) 
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Validation  Experiment  Types 

•  System  Validation: 

-  Combination  of  advanced  technologies  that  are  integrated 
into  a  system  to  validate  a  single  technological  capability: 

•  Solar  Electric  Propulsion  -  “Mission  Defining” 


Deep  Space  1 
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Validation  Experiment  Types 


Subsystem: 

Standalone  subsystem  experiment  that  can  be  flown  as  a  payload 
on  a  variety  of  space-borne  platforms 

Multiple  experiments  on  single  spacecraft  not  dependent  upon 
each  other  for  validation 
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Validation  of  a  Technology  Advance 
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-Provide  a  capability  needed  by 
NASA 

-  Provide  a  significant  step  toward 
realizing  that  capability 

-Require  in-space  testing  for 
proper  validation 

Validation  is 
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•  Empirical  evidence:  test  data 
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replicate/predict  test  results 
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Technological  Maturity  Requirements 


•  NMP  projects  are  flight  missions 

-Candidate  technology  advances  must  be  at  TRL  >  4  at  start  of  Phase  B 

-Technology  maturation  plan  is  part  of  evaluation  for  selection  for 
Phase  B 

-Technologies  must  reach  maturity  =  TRL  6  by  start  of  start  of  system 
assembly,  test  and  launch  operations  (ATLO) 

•  Increasing  technological  maturity  in  the  project  development 
environment  is  a  defining  element  of  NMP  projects 

-  Required  level  of  maturity  for  each  stage  must  be  defined 

-  Fair,  well  understood  criteria  for  determining  the  level  of  maturity  must 
be  defined 

-An  objective  method  for  assessing  the  degree  to  which  the  criteria 
have  been  satisfied  must  be  established  and  implemented 

•  NMP  is  strongly  motivated  to  establish  meaningful  and  realistic 
technology-specific  criteria  by  which  to  judge  the  maturity  of  a 
technology  advance 
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Cost  and  Schedule  Risk 


•  NMP  missions  are  cost  constrained 

-  Acute  sensitivity  to  project  cost  and  schedule  risk  must 
be  maintained  throughout  an  NMP  project 

-  Risk-based  cost  and  schedule  reserves  are  required  at 
Phase  A-B  transition  and  at  project  Confirmation  (start 
of  Phase  C) 

-  Proceeding  to  a  subsequent  project  development  phase 
before  the  technologies  are  sufficiently  mature  places 
the  project  at  high  cost  and  schedule  risk  and  project 
reserves  are  likely  to  be  exceeded 

•  Leads  to  a  reduction  in  scope  of  the  validation  experiment  and 
reduces  the  value  of  the  project  to  NASA 
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Technical,  Cost  and  Schedule  Risks 
in  Project  Lifecycles 
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Increasing  Technological  Maturity 
in  a  Project  Environment 


The  risk  of  spiraling  cost  growth  associated  with 
maturing  a  technology  advance  in  a  New  Millennium 
project  is  mitigated  by  defining  clearly  at  the  outset: 

-  The  capability  to  be  put  in  place  based  on  the  needs  of 
future  SMD  science  missions 

-  The  technology  advance  that  can  provide  the  capability 

-  The  level  of  maturity  to  be  achieved  by  that  technology 
advance  at  each  stage  of  the  project 


Phase  A 


Phase  B  Phase  C/D  Phase  E 


Formulation 

Refinement 


Implementation 


Operations 


AFRL  TMC  2007 


16 


NMP’s  Approach  to  Assessing 
Technological  Maturity  Begins  with  Guidelines 


Guidelines 

-  Establish  general  criteria  for  technological  maturity  at 
each  TRL 

•  Provides  broad  uniformity  across  the  program 

-  Assess  Technological  Maturity  in  Three  Areas 

•  Configuration:  increasing  maturity  approaches  that  of  a  flight 
system 

•  Models:  increased  fidelity  and  complexity  indicates  higher 
maturity 

•  Test  environments:  greater  realism  that  increasingly  stresses 
the  technology  advance  demonstrates  greater  maturity 

-  Leave  some  elements  of  each  criteria  undefined 

•  Specific  test  article  •  Specific  test  environment 

•  Degree  of  model  fidelity 
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Assessing  Technological  Maturity 
Requires  Clear  Criteria 

*  NMP  employs  the  TRLs  as  a  “measure”  of 
technological  maturity 

*  Criteria  are  developed  that  allow  maturity  to  be 
assessed  as  the  project  unfolds 

-  Developed  as  a  joint  activity  by  the  provider  of  the 
technology  advance,  the  project,  and  the  program  office 

-  Developed  early  in  the  project  to  allow  adequate  planning 

-  Revisited  throughout  the  project’s  life  to  allow  for 
exogenous  changes 

-  Are  appropriate  to  the  associated  project  phase  to  prevent 
the  project  from  incurring  unwarranted  risk 

•  NMP  Projects  invest  relatively  heavily  in  Phases  A  &  B 

*  Assessment  process  must  be,  and  be  seen  to  be, 
objective  and  equitable 

AFRLTMC2007  18 


Technology  Review  Board  (TRB) 

Jointly  Develops  Assessment  Criteria  and  Assesses 

Technological  Maturity 

•  Assessment  by  team  of  independent,  subject-matter 
experts  selected  for  each  technology  advance 

-  Specific  criteria  established  early 

•  Based  on  the  guidelines  as  a  starting  point 

•  Developed  jointly  with  the  Technology  Provider 

•  Developed  with  the  needs  of  the  likely  first  user  in  mind 

•  Used  for  schedule  and  budget  planning 

-  Assessment  based  on  jointly  developed  criteria 

•  Independent  team  provides  flexibility  while  retaining  focus 
on  value 

-  Responses  to  inevitable  exogenous  changes  retain  focus  on 
value  for  the  ultimate  user 

•  Both  Technology  Provider  and  subject-matter  experts  are  interested  in 
successful  validation 


Guidelines 


Technology  Independent  Validation 

Provider  ^BB  Experts  ^B^B,  Criteria  &  Plan 
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The  Technology  Provider  and  the 
Technology  Review  Board  Work  in  Concert 
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For  ST8  &  ST9  the  TRB  was  Integral  to 
Selection  and  Implementation 
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Summary 

Key  Points  about  NMP’s  Approach  to 
Technology  Maturation 

*  Technology  advances  to  be  matured  are  selected  in 
response  to  user-identified,  needed  capabilities 

*  New  Millennium  missions  are  a  lower  cost  method  of 
maturing  these  technology  advances  than  doing  so  as 
part  of  a  science  mission 

*  Maturing  the  technology  advance  consists  of 

-  Test  articles  of  continuously  increasing  fidelity 

-  Test  environments  and  tests  that  increasingly  stress  the 
technology  advance 

-  Models  of  the  technology  advance  that  replicate  the  test  data 

*  Clear  guidelines  identify  what  is  expected 

*  Independent  experts  work  with  the  technology  provider 
to  identify  mutually  acceptable  criteria  for  assessing 
the  maturity  of  the  technology  advance 
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